Maximum Likelihood Estimates Of Parameters Research Articles

Applications of nature-inspired metaheuristic algorithms for tackling optimization problems across disciplines

Nature-inspired metaheuristic algorithms are important components of artificial intelligence, and are increasingly used across disciplines to tackle various types of challenging optimization problems. This paper demonstrates the usefulness of such algorithms for solving a variety of challenging optimization problems in statistics using a nature-inspired metaheuristic algorithm called competitive swarm optimizer with mutated agents (CSO-MA). This algorithm was proposed by one of the authors and its superior performance relative to many of its competitors had been demonstrated in earlier work and again in this paper. The main goal of this paper is to show a typical nature-inspired metaheuristic algorithmi, like CSO-MA, is efficient for tackling many different types of optimization problems in statistics. Our applications are new and include finding maximum likelihood estimates of parameters in a single cell generalized trend model to study pseudotime in bioinformatics, estimating parameters in the commonly used Rasch model in education research, finding M-estimates for a Cox regression in a Markov renewal model, performing matrix completion tasks to impute missing data for a two compartment model, and selecting variables optimally in an ecology problem in China. To further demonstrate the flexibility of metaheuristics, we also find an optimal design for a car refueling experiment in the auto industry using a logistic model with multiple interacting factors. In addition, we show that metaheuristics can sometimes outperform optimization algorithms commonly used in statistics.

Functional mixtures-of-experts

AbstractWe consider the statistical analysis of heterogeneous data for prediction, in situations where the observations include functions, typically time series. We extend the modeling with mixtures-of-experts (ME), as a framework of choice in modeling heterogeneity in data for prediction with vectorial observations, to this functional data analysis context. We first present a new family of ME models, named functional ME (FME), in which the predictors are potentially noisy observations, from entire functions. Furthermore, the data generating process of the predictor and the real response, is governed by a hidden discrete variable representing an unknown partition. Second, by imposing sparsity on derivatives of the underlying functional parameters via Lasso-like regularizations, we provide sparse and interpretable functional representations of the FME models called iFME. We develop dedicated expectation–maximization algorithms for Lasso-like regularized maximum-likelihood parameter estimation strategies to fit the models. The proposed models and algorithms are studied in simulated scenarios and in applications to two real data sets, and the obtained results demonstrate their performance in accurately capturing complex nonlinear relationships and in clustering the heterogeneous regression data.

Mixtures of regressions using matrix-variate heavy-tailed distributions

AbstractFinite mixtures of regressions (FMRs) are powerful clustering devices used in many regression-type analyses. Unfortunately, real data often present atypical observations that make the commonly adopted normality assumption of the mixture components inadequate. Thus, to robustify the FMR approach in a matrix-variate framework, we introduce ten FMRs based on the matrix-variate t and contaminated normal distributions. Furthermore, once one of our models is estimated and the observations are assigned to the groups, different procedures can be used for the detection of the atypical points in the data. An ECM algorithm is outlined for maximum likelihood parameter estimation. By using simulated data, we show the negative consequences (in terms of parameter estimates and inferred classification) of the wrong normality assumption in the presence of heavy-tailed clusters or noisy matrices. Such issues are properly addressed by our models instead. Additionally, over the same data, the atypical points detection procedures are also investigated. A real-data analysis concerning the relationship between greenhouse gas emissions and their determinants is conducted, and the behavior of our models in the presence of heterogeneity and atypical observations is discussed.

Inference for Parameters of Exponential Distribution under Combined Type II Progressive Hybrid Censoring Scheme

In recent years, various forms of progressive hybrid censoring schemes (PHCS) have gained significant traction in survival and reliability analysis studies due to their versatility. However, these PHCS variants are often characterized by complexity stemming from the multitude of parameters involved in their specification. Consequently, the primary objective of this paper is to propose a unified approach termed combined type II progressive hybrid censoring scheme (ComT2PHCS) capable of encompassing several existing PHCS variations. Our analysis focuses specifically on the exponential distribution (ExDist). Bayesian inference techniques are employed to estimate the parameters of the ExDist under the ComT2PHCS. Additionally, we conduct fundamental distributional analyses and likelihood inference procedures. We derive the conditional moment-generating function (CondMGF) of maximum likelihood estimator (MLE) for parameters of the ExDist under ComT2PHCS. Further, we use CondMGF for the distribution of MLE for parameters of ExDist under ComT2PHCS. Finally, we provide an illustrative example to elucidate the inference methods derived in this paper.

Replica analysis of overfitting in regression models for time to event data: the impact of censoring

We use statistical mechanics techniques, viz. the replica method, to model the effect of censoring on overfitting in Cox’s proportional hazards model, the dominant regression method for time-to-event data. In the overfitting regime, Maximum Likelihood (ML) parameter estimators are known to be biased already for small values of the ratio of the number of covariates over the number of samples. The inclusion of censoring was avoided in previous overfitting analyses for mathematical convenience, but is vital to make any theory applicable to real-world medical data, where censoring is ubiquitous. Upon constructing efficient algorithms for solving the new (and more complex) Replica Symmetric (RS) equations and comparing the solutions with numerical simulation data, we find excellent agreement, even for large censoring rates. We then address the practical problem of using the theory to correct the biased ML estimators without knowledge of the data-generating distribution. This is achieved via a novel numerical algorithm that self-consistently approximates all relevant parameters of the data generating distribution while simultaneously solving the RS equations. We investigate numerically the statistics of the corrected estimators, and show that the proposed new algorithm indeed succeeds in removing the bias of the ML estimators, for both the association parameters and for the cumulative hazard.

Two-Stage Limited-Information Estimation for Structural Equation Models of Round-Robin Variables

We propose and demonstrate a new two-stage maximum likelihood estimator for parameters of a social relations structural equation model (SR-SEM) using estimated summary statistics (Σ^) as data, as well as uncertainty about Σ^ to obtain robust inferential statistics. The SR-SEM is a generalization of a traditional SEM for round-robin data, which have a dyadic network structure (i.e., each group member responds to or interacts with each other member). Our two-stage estimator is developed using similar logic as previous two-stage estimators for SEM, developed for application to multilevel data and multiple imputations of missing data. We demonstrate out estimator on a publicly available data set from a 2018 publication about social mimicry. We employ Markov chain Monte Carlo estimation of Σ^ in Stage 1, implemented using the R package rstan. In Stage 2, the posterior mean estimates of Σ^ are used as input data to estimate SEM parameters with the R package lavaan. The posterior covariance matrix of estimated Σ^ is also calculated so that lavaan can use it to calculate robust standard errors and test statistics. Results are compared to full-information maximum likelihood (FIML) estimation of SR-SEM parameters using the R package srm. We discuss how differences between estimators highlight the need for future research to establish best practices under realistic conditions (e.g., how to specify empirical Bayes priors in Stage 1), as well as extensions that would make 2-stage estimation particularly advantageous over single-stage FIML.

Inference on order restricted means of inverse Gaussian populations under heteroscedasticity

The hypothesis testing problem of homogeneity of k(≥2) inverse Gaussian means against ordered alternatives is studied when nuisance or scale-like parameters are unknown and unequal. The maximum likelihood estimators (MLEs) of means and scale-like parameters are obtained when means satisfy some simple order restrictions and scale-like parameters are unknown and unequal. An iterative algorithm is proposed for finding these estimators. It has been proved that under a specific condition, the proposed algorithm converges to the true MLEs uniquely. A likelihood ratio test and two simultaneous tests are proposed. Further, an algorithm for finding the MLEs of parameters is given when means are equal but unknown. Using the estimators, the likelihood ratio test is developed for testing against ordered alternative means. Using the asymptotic distribution, the asymptotic likelihood ratio test is proposed. However, for small samples, it does not perform well. Hence, a parametric bootstrap likelihood ratio test (PB LRT) is proposed. Therefore, the asymptotic validity of the bootstrap procedure has been shown. Using the Box-type approximation method, test statistics are developed for the two-sample problem of equality of means when scale-like parameters are heterogeneous. Using these, two PB-based heuristic tests are proposed. Asymptotic null distributions are derived and PB accuracy is also developed. Two asymptotic tests are also proposed using the asymptotic null distributions. To get the critical points and test statistics of the three PB tests and two asymptotic tests, an ‘R’ package is developed and shared on GitHub. Applications of the tests are illustrated using real data.

Parameter estimation with reluctant quantum walks: a maximum likelihood approach

The parametric maximum likelihood estimation problem is addressed in the context of quantum walk theory for quantum walks on the lattice of integers. A coin action is presented, with the real parameter θ to be estimated identified with the angular argument of an orthogonal reshuffling matrix. We provide analytic results for the probability distribution for a quantum walker to be displaced by d units from its initial position after k steps. For k large, we show that the likelihood is sharply peaked at a displacement determined by the ratio d/k which is correlated with the reshuffling parameter θ. We suggest that this ‘reluctant walker’ behaviour provides the framework for maximum likelihood estimation analysis, allowing for robust parameter estimation of θ via return probabilities of closed evolution loops and quantum measurements of the position of quantum walker with ‘reluctance index’ r = d/k.

Interval estimation of relative risks for combined unilateral and bilateral correlated data

ABSTRACT Measurements are generally collected as unilateral or bilateral data in clinical trials, epidemiology, or observational studies. For example, in ophthalmology studies, the primary outcome is often obtained from one eye or both eyes of an individual. In medical studies, the relative risk is usually the parameter of interest and is commonly used. In this article, we develop three confidence intervals for the relative risk for combined unilateral and bilateral correlated data under the equal dependence assumption. The proposed confidence intervals are based on maximum likelihood estimates of parameters derived using the Fisher scoring method. Simulation studies are conducted to evaluate the performance of proposed confidence intervals with respect to the empirical coverage probability, the mean interval width, and the ratio of mesial non-coverage probability to the distal non-coverage probability. We also compare the proposed methods with the confidence interval based on the method of variance estimates recovery and the confidence interval obtained from the modified Poisson regression model with correlated binary data. We recommend the score confidence interval for general applications because it best controls converge probabilities at the 95% level with reasonable mean interval width. We illustrate the methods with a real-world example.

Stochastic DDM with regime–switching process

This paper introduces a stochastic dividend discount model with regime–switching and conditional heteroscedasticity. We obtain formulas, corresponding to a default probability and $ \ell $-th moment of a random stock price of a company. Next, we connect option pricing, hedging, and equity–linked life insurance products to the stochastic dividend discount model. Also, the paper provides maximum likelihood estimators of parameters of the stochastic dividend discount model. Finally, we present numerical results for the equity–linked life insurance products in the case of Walmart.

Statistical inference for a competing failure model based on the Wiener process and Weibull distribution.

Competing failure models with degradation phenomena and sudden failures are becoming more and more common and important in practice. In this study, the generalized pivotal quantity method was proposed to investigate the modeling of competing failure problems involving both degradation and sudden failures. In the competing failure model, the degradation failure was modeled through a Wiener process and the sudden failure was described as a Weibull distribution. For point estimation, the maximum likelihood estimations of parameters $ \mu $ and $ \sigma^2 $ were provided and the inverse estimation of parameters $ \eta $ and $ \beta $ were derived. The exact confidence intervals for parameters $ \mu $, $ \sigma^2 $, and $ \beta $ were obtained. Furthermore, the generalized confidence interval of parameter $ \eta $ was obtained through constructing the generalized pivotal quantity. Using the substitution principle, the generalized confidence intervals for the reliability function, the $ p $th percentile of lifetime, and the mean time to failure were also obtained. Simulation technique was carried out to evaluate the performance of the proposed generalized confidence intervals. The simulation results showed that the proposed generalized confidence interval was effective in terms of coverage percentage. Finally, an example was presented to illustrate the application of the proposed method.

Drift Estimation of the Threshold Ornstein-Uhlenbeck Process From Continuous and Discrete Observations

We refer by threshold Ornstein-Uhlenbeck to a continuous-time threshold autoregressive process. It follows the Ornstein-Uhlenbeck dynamics when above or below a fixed level, yet at this level (threshold) its coefficients can be discontinuous. We discuss (quasi)-maximum likelihood estimation of the drift parameters, both assuming continuous and discrete time observations. In the ergodic case, we derive consistency and speed of convergence of these estimators in long time and high frequency. Based on these results, we develop a test for the presence of a threshold in the dynamics. Finally, we apply these statistical tools to short-term US interest rates modeling.

Combined with linear regression analysis of Hainan coconut carving art dissemination path analysis

Abstract Hainan coconut carving is regarded as an excellent traditional cultural treasure of the Chinese nation, which has a long history with a long history, but the current dissemination of the art of Hainan coconut carving is faced with the dilemma of decline. This paper carries out the construction of a multiple linear regression model, using the least squares method to determine the regression parameters and find the maximum likelihood estimation of the regression parameters. Then, the relevant elements of the art communication path, based on the elements of the construction of the art communication of multiple linear regression model and the production of questionnaires used to obtain the art communication path influence factors data. The least squares and maximum likelihood parameter estimation are numerically simulated, and the art communication path is analyzed using multiple linear regression. The results show that the mean square error of the least squares method is below 0.001 when the sample size of maximum likelihood estimation is increased from 100 to 300, the error rate of different parameters is reduced by 70.83%, and the significance of the influencing factors in the multivariate linear regression model is less than 0.05. Based on the linear regression, we can analyze the influencing factors of the dissemination path of the art of Hainan Coconut Sculpture, which provides a reference to the further expansion of the art of Hainan Coconut Sculpture. It can provide a reference for further expanding the dissemination of Hainan coconut carving art.

Better performance for right-skewed data using an alternative gamma model

BackgroundThe Maximum Likelihood Estimator (MLE) for parameters of the gamma distribution is commonly used to estimate models of right-skewed variables such as costs, hospital length of stay, and appointment wait times in Economics and Healthcare research. The common specification for this estimator assumes the variance is proportional to the square of the mean, which underlies estimation and specification tests. We present a specification in which the variance is directly proportional to the mean.MethodsWe used simulation experiments to investigate finite sample results, and we used United States Department of Veterans Affairs (VA) healthcare cost data as an empirical example comparing the fit and predictive ability of the models.ResultsSimulation showed the MLE based on a correctly specified alternative has less parameter bias, lower standard errors, and less skewness in distribution than a misspecified standard model. The application to VA healthcare cost data showed the alternative specification can have better R square, smaller root mean squared error, and smaller mean residuals within deciles of predicted values.ConclusionsThe alternative gamma specification can be a useful alternative to the standard specification for estimating models of right-skewed continuous variables.

Inference for dependent complementary competing risks model from an inverted Kumaraswamy distribution under ranked set sampling

AbstractRanked set sampling (RSS) has been proved to be an efficient sampling design for parametric and nonparametric inference. This paper explores inference for a maximum RSS procedure with unequal samples (MRSSU) with multiple dependent failure causes. When the lifetimes of units are characterized by a proposed complementary competing risks model, classical likelihood and Bayesian approaches are discussed for parameters and reliability estimation. Maximum likelihood estimators of model parameters and associated existence and uniqueness are established, and approximate confidence intervals are constructed using asymptotic theory and delta methods. With respect to general flexible priors, Bayesian estimates of interested quantities are also performed and a Monte Carlo sampling algorithm is proposed for complex posterior computation. Additionally, when extra historical information between the competing risks parameters is available, likelihood and Bayesian estimation are also studied under an order restriction case. Different methods are compared based on extensive simulation studies, and another real data example is demonstrated for application propose.

Weighted Prakaamy Distribution: Properties, Applications and Analysis

The paper introduces a new probability distribution, which is weighted version of the Prakaamy distribution. The paper explores various statistical properties of the Weighted Prakaamy (WP)distribution including probability density function (PDF), cumulative distribution function (CDF), moments, moment generating function, characteristics function, reliability analysis, ordered statistics, maximum likelihood estimation of parameters, entropies, likelihood ratiotest, and Bonferroni and Lorenz curves. The paper uses simulations to evaluate the performance of maximum likelihood estimators. The authorsapply the WP distribution to various real-life data sets from fields of engineering and medical science. This empirical analysis aims to evaluate the performance of the distribution in modeling and predicting real-world phenomena. The paper suggests that WP distribution outperforms other probability distributions including Prakaamy distribution, Exponential distribution, Erlang Truncated Exponential distribution, Power Lindley distribution and Lindley distribution.

A New Version of Prakaamy Distribution with Properties and Applications

This paper introduces a novel statistical model, the Length-Biased version of the Prakaamy distribution, characterized by one parameter. The study presents a comprehensive analysis of this distribution, including the derivation of its probability density function (PDF), cumulative distribution function (CDF), moments, moment generating function, characteristics function, reliability analysis, ordered statistics, maximum likelihood estimation of parameter, entropies, likelihood ratio test, and Bonferroni and Lorenz curves.  To validate the effectiveness of the proposed model, it is applied to two real-life time data sets, demonstrating its applicability and utility in practical scenarios.

The Concavity of Conditional Maximum Likelihood Estimation for Logit Panel Data Models with Imputed Covariates

In estimating logistic regression models, convergence of the maximization algorithm is critical; however, this may fail. Numerous bias correction methods for maximum likelihood estimates of parameters have been conducted for cases of complete data sets, and also for longitudinal models. Balanced data sets yield consistent estimates from conditional logit estimators for binary response panel data models. When faced with a missing covariates problem, researchers adopt various imputation techniques to complete the data and without loss of generality; consistent estimates still suffice asymptotically. For maximum likelihood estimates of the parameters for logistic regression in cases of imputed covariates, the optimal choice of an imputation technique that yields the best estimates with minimum variance is still elusive. This paper aims to examine the behaviour of the Hessian matrix with optimal values of the imputed covariates vector, which will make the Newton–Raphson algorithm converge faster through a reduced absolute value of the product of the score function and the inverse fisher information component. We focus on a method used to modify the conditional likelihood function through the partitioning of the covariate matrix. We also confirm that the positive moduli of the Hessian for conditional estimators are sufficient for the concavity of the log-likelihood function, resulting in optimum parameter estimates. An increased Hessian modulus ensures the faster convergence of the parameter estimates. Simulation results reveal that model-based imputations perform better than classical imputation techniques, yielding estimates with smaller bias and higher precision for the conditional maximum likelihood estimation of nonlinear panel models.

On the Kolmogorov Distance for the Maximum Likelihood Estimator in the Explosive Ornstein-Uhlenbeck Process

The paper estimates the Kolmogorov distance between the distribution of the normalized maximum likelihood estimator of the positive drift parameter in the nonergodic Ornstein-Uhlenbeck process and the standard Cauchy distribution and shows exponential error rate for large time limit.

Asymptotic Predictive Inference of Negative Lower Tail Index Distributions

ABSTRACT In this paper, the results of El-Adll et al. [Asymptotic prediction for future observations of a random sample of unknown continuous distribution, Complexity 2022 (2022), Art. ID 4073799], are extended to the lower negative tail index distributions. Three distinct estimators of the lower negative tail index are proposed, as well as an asymptotic confidence interval. Moreover, different asymptotic predictive intervals for future observations are constructed for distributions attracted to the lower extreme value distribution with a negative tail index. Furthermore, the asymptotic maximum likelihood estimator (AMLE) of the shape parameter, as well as an asymptotic maximum likelihood predictor (AMLP), are obtained. Finally, extensive simulation studies are conducted to demonstrate the efficiency of the proposed methods.